This invention relates to a method and apparatus for orthographic error correction using phonetic transcription.
An existing algorithm that has proven very successful in spell correction of natural language text is edit distance, which involves changing the text of an incorrectly spelled word using a minimal number of possible modifications, in order to find a correctly spelled alternative. The correctly spelled word with the lowest conversion cost, based on the number of types of modification applied, is typically considered the most likely correction.
In addition to edit distance, many spell correction approaches try to correct phonetically sourced spelling mistakes, usually by substituting portions of the incorrect word with a phonetically equivalent sequence of characters, until a correct spelling is located. For example: “ee”==“ea” as in “seen” not “sean”; “ay”==“ae” as in “say” not “sae”; “uff”==“ough” as in “scuff” not “skough”; and “ow”==“ough” as in “now” not “nough”. Other methods have used acoustic hidden markov models with the similar data.
This type of processing is performed using a set of well-documented phonetic pairs, similar to those described above, which map sequences of letters that have a similar phonetic quality, or which may be described by the same phonemes in certain words. While this can be helpful in correcting errors produced by speakers who perform a similar but erroneous substitution in converting from spoken to written form (as when ‘photograph’ in English is misspelled as ‘fotograf’), these methods are severely limited by the dependence on the knowledge of language-specific pairs of common phonetic mistakes by human authors.
While these methods have been relatively successful, there are an increasing number of cases that require a more robust method for recognizing and correcting phonetically sourced spelling errors. Scenarios such as a single piece of text combining multiple languages, or errors in output from speech to text systems, have provided a much more challenging environment for spell correction, and methods that focus on common language-specific phonetic error patterns, in human authored text, are simply not good enough.
Phonetic alphabets that allow the transcription of the sounds of human language have been developed in order to allow linguists to document human utterances in a consistent fashion. In these alphabets a single character or symbol represents a single phoneme or phonetic unit of sound. An example of such an alphabet is the International Phonetic Alphabet (IPA).
In rare cases this type of alphabet is used to write down the utterances of languages that do not have an alternative written form. There are distinct advantages in considering such resources as an alternative writing system for the purposes of NLP, and particularly for spell correction.